Video picture format conversion method and corresponding device

ABSTRACT

The present invention can be implemented in a video coder or decoder or directly in any type of display device. According to the invention, a distance representative of the edge orientation in the input picture is calculated, for at least one point situated inside a zone delimited by a first set of neighbouring pixels of the input picture, it is calculated independently from the grid of pixels of the output picture. For at least one pixel of the output picture situated in this zone, a second set of pixels is determined in the input picture from the distance and the position of said pixel of the output picture in this zone. The value of this output pixel is then determined from the value of the pixels of said second set of pixels. This method enables a reduction in the number of calculations for this format conversion.

FIELD OF THE INVENTION

The present invention relates to a method and a device for video pictureformat conversion. It can be implemented in a video coder or decoder ordirectly in any type of display device.

TECHNOLOGICAL BACKGROUND OF THE INVENTION

There are currently two formats associated with standard HDTV (HighDefinition TV). These two formats are the 720p format and the 1080iformat. The 720p format produces progressive pictures comprising 720lines and 1280 columns and the 1080i format produces interlaced picturescomprising 1080 lines and 1920 columns. Each of these formats hasadvantages that are specific to it. The 720p format presents a bettertemporal resolution and reproduces fast moving objects without creatinga blur effect while the 1080i format presents a better spatialresolution. The programme broadcasters have chosen to use one or otherof these formats but not both together. There is therefore a realnecessity to process, prior to display, the format of broadcastprogrammes, to transform it into a format supported by the displaydevice used to display said programmes.

It is known in the art to change the format of a video picture by firstcarrying out a detection of the local edge orientation in the inputpicture then carrying out an interpolation based on the detectedorientations. A distance representative of the edge orientation iscalculated at a point of the input picture having the same spatialposition (i.e. the same spatial coordinates) as the pixel of the outputpicture to be interpolated and this distance is then used to calculatethe value of this pixel of the output picture. This method is relativelycomplex and consuming in terms of calculation resources.

SUMMARY OF THE INVENTION

The purpose of the present invention is to improve this method.

The present invention relates to a format conversion method of an inputpicture to an output picture, comprising the following steps:

-   -   calculate, in at least one point situated inside a zone        delimited by a first set of neighbouring pixels of the input        picture, a distance representative of the edge orientation at        this point,    -   determine, for at least one pixel situated in a zone of the        output picture corresponding to the zone delimited by the first        set of neighbouring pixels of said input picture, a second set        of pixels in the input picture from the calculated distance for        said point in the zone and from the position of said pixel of        the output picture in the zone of the output picture,    -   determine the value of said pixel of the output picture from the        value of the pixels of the second set of pixels.

According to a particular embodiment, the neighbouring pixels of thefirst set belong to at least two consecutive lines or columns of pixelsof the input picture.

The calculation of the distance representative of the edge orientationis carried out on a single point of the zone of the input picture toreduce the number of calculations. This point is thus advantageouslysituated at the centre of said zone.

According to the invention, the distance representative of the edgeorientation at a point of the input picture depends on a local gradientof video components of the input picture at said point.

According to a particular embodiment, if the calculated distance exceedsa predefined maximum value, the distance is capped at said maximumvalue.

According to another particular embodiment, if the calculated distanceexceeds a predefined maximum value, the distance is adjusted to a nullvalue.

According to the invention, the value of the pixel of the output pictureis advantageously determined by bilinear interpolation of pixels of thesecond set.

The invention also relates to a device for format conversion of an inputpicture into an output picture, the input picture and the output picturecomprising the pixels, comprising:

-   -   a calculation circuit to calculate, in at least one point        situated inside a zone delimited by a first set of neighbouring        pixels of the input picture, a distance representative of the        edge orientation at this point, and    -   an interpolation circuit to determine, for at least one pixel        situated in a zone of the output picture corresponding to the        zone delimited by the first set of neighbouring pixels of said        input picture, a second set of pixels in the input picture from        the distance or distances calculated for the point or points of        the zone and from the position of the pixel of the output        picture in the zone of the output picture, and to determine the        value of the pixel of said output picture from the value of the        pixels of the second set of pixels of the input picture.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood upon reading the followingdescription, provided as a non-restrictive example and referring to theannexed drawings wherein:

FIG. 1 shows the steps in a first embodiment of the method according tothe invention

FIG. 2 shows the position of a point P of the input picture in which itis calculated at a distance representative of the edge orientation,

FIG. 3 shows the parameters used to calculate the value of a pixelP_(out) of the output picture located in the same zone of the picture asthe pixel P of the input picture shown in FIG. 2,

FIG. 4 shows a device capable of implementing the method of FIG. 1,

FIG. 5 shows the steps in a second embodiment of the method according tothe invention, and

FIG. 6 shows a device capable of implementing the method of FIG. 5.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION

The method of the invention is used to carry out a format conversion ofan input picture. The method produces an output picture having a formatdifferent to that of the input picture, the output picture comprising alarger or smaller number of lines and/or columns than the input picture.The lines and/or columns of the output picture are generated byinterpolation of the lines and/or columns of the input picture.

According to the invention, the interpolation is an “edge orientatedinterpolation”. This interpolation is more particularly realised whenbased on a detection of local edge orientations in the input picture.This detection is carried out by calculating the distancesrepresentative of the orientation of these edges. An example of thecalculation of this distance will be described later in the description.

FIG. 1 shows the steps in a first embodiment of the method according tothe invention. The method is applied to all or part of an input pictureto generate an output picture of a different format. In reference toFIG. 1, the method comprises a first step 110 of calculation of adistance representative of an edge orientation and at least one pointsituated inside a zone delimited by a first set of neighbouring pixelsof the input picture. This zone is for example delimited by 4 adjacentpixels belonging to two consecutive lines and two consecutive columns ofthe input picture. According to the invention, the distance calculatedfor a point in this zone of the input picture is used to calculate thevalue of the pixels of the output picture present in this zone of thepicture.

An example of the calculation of the representative distance d of theedge orientation at a point of the input picture is described hereafter.This distance d, is calculated at a point P of the input picture asshown in FIG. 2. FIG. 2 shows 8 pixels of the input picture spread overtwo consecutive lines y_(n) and y_(n)+1 of the input picture. Thesepixels belong to the columns of pixels numbered x_(m)−1, x_(m), x_(m)+1and x_(m)+2 of the input picture. The pixels of the input picture arerepresented on the figure by black dots. The distance is calculated at apoint P situated inside the zone delimited by the coordinates(x_(m),y_(n)), (x_(m)+1,y_(n)), (x_(m)+1,y_(n)+1) and (x_(m),y_(n)+1).The pixel P is identified by the coordinates (x′,y′) in the inputpicture. In this example, the point P is situated at the centre of thezone delimited by the pixels at coordinates (x_(m),y_(n)),(x_(m)+1,y_(n)), (x_(m)+1,y_(n)+1) and (x_(m),y_(n)+1). A horizontalgradient, noted as I_(x)(x′,y′), and a vertical gradient, noted asI_(y)(x′,y′), are calculated for the luminance video component (Y in thespace of the colours [Y,Cb,Cr] or [Y,U,V]) at this point P.

${I_{x}\left( {x^{\prime},y^{\prime}} \right)} = {{\frac{\partial I}{\partial x}\left( {x^{\prime},y^{\prime}} \right)} = {{I\left( {{x_{m} + 2},y_{n}} \right)} + {I\left( {{x_{m} + 2},{y_{n} + 1}} \right)} - {I\left( {{x_{m} - 1},y_{n}} \right)} - {I\left( {{x_{m} - 1},{y_{n} + 1}} \right)}}}$${I_{y}\left( {x^{\prime},y^{\prime}} \right)} = {{\frac{\partial I}{\partial y}\left( {x^{\prime},y^{\prime}} \right)} = {{I\left( {x_{m},{y_{n} + 1}} \right)} + {I\left( {{x_{m} + 1},{y_{n} + 1}} \right)} - {I\left( {x_{m},y_{n}} \right)} - {I\left( {{x_{m} + 1},y_{n}} \right)}}}$

where I(x,y) designates the luminance component of the pixel atcoordinates (x,y).

The distance d to point P is then taken to equal:

$d = \frac{I_{y}\left( {x^{\prime},y^{\prime}} \right)}{I_{x}\left( {x^{\prime},y^{\prime}} \right)}$

The distance d calculated for the pixel P is shown in FIG. 2. A linesegment S links the point at coordinates (x′+d,y_(n)) and the point atcoordinates (x′−d,y_(n)+1).

According to the invention, this distance d calculated at the point P isused to determine the value of all the pixels of the output picturebelonging to the zone of the output picture corresponding to the zonedelimited by the points at coordinates (x_(m),y_(n)), (x_(m)+1,y_(n)),(x_(m)+1,y_(n)+1) and (x_(m),y_(n)+1) in the input picture. It should benoted that the coordinates (x_(m),y_(n)), (x_(m)+1,y_(n)),(x_(m)+1,y_(n)+1), (x_(m),y_(n)+1) are the coordinates in the inputpicture. The output picture having a number of columns and a number oflines different to those of the input picture, the pixels delimitingthis zone have different coordinates in the output picture. Naturally itis possible to extend this zone to any zone surrounding the point Phaving a different size and/or form.

The format conversion modifies the number and position of lines and/orcolumns of pixels of the picture. For example, in the output picture, aline y″ of pixels is present between the lines y_(n) and y_(n)+1 of theinput picture and a column of pixels x″ is present between the columnsof pixels x_(m) and x_(m)+1 of the input picture. A pixel P_(out) of theoutput picture is present at the intersection of the line y″ and thecolumn x″ as shown in FIG. 3. The coordinates (x″,y″) are thecoordinates of the pixel P_(out) in the input picture. The outputpicture having a number of columns and/or lines different to those ofthe input picture, the pixel P_(out) has different coordinates, noted as(x_(out), y_(out)), in the output picture. The distance d calculated forthe point P is used to determine the values of all the pixels P_(out)present in the zone delimited by the points at coordinates(x_(m),y_(n)), (x_(m)+1,y_(n)), (x_(m)+1,y_(n)+1) and (x_(m),y_(n)+1).

With reference again to FIG. 1, the method also comprises adetermination step 120, for at least one pixel of the output picturesituated in this zone, of a second set of pixels in the input picturefrom the distance d previously calculated and from the position of thispixel of the output picture in this zone. For example, a second set ofpixels of the input picture for the pixel P_(out) is thus calculated.This step is described with reference to FIG. 3. To improveunderstanding of this step, the following notations are used.

-   -   ν is the ratio of the number of lines of the input picture to        the number of lines of the output picture,    -   r is the ratio of the number of columns of the input picture to        the number of columns of the output picture,        For the pixel P_(out) at coordinates (x_(out),y_(out)) in the in        the output picture, the lines of pixels y_(n) and y_(n)+1 of the        input picture closest to the line y_(out) are determined.

y″=ν·y _(out)

y _(n) =E(y″)

-   -   where E(x) designates the integer part of the variable x.

A vertical phase φ=|y″−y_(n)| representing the distance between the liney″ of the pixel P_(out) and the line y_(n) of the input picture isdefined. φ is comprised between 0 and 1.

The columns of pixels x_(m) and x_(m)+1 of the input picture closest tothe column x_(out) are also determined.

X″=r·x _(out)

x _(m) =E(x″)

A horizontal phase θ=x″−x_(m)−0.5 representing the distance between theline x′ of the pixel P_(out) and the pixel P of the input picture isdefined. θ is comprised between −0.5 and 0.5.

If the line segment S is translated horizontally so that it passesthrough the pixel P_(out), this segment then links the point atcoordinates (x_(yn),y_(n)) and the point at coordinates(x_(yn+1),y_(n)+1).

Then x_(yn)=x″+2·φ·d+θ=x″+d_(sup) and x_(yn+1)=x″+2·(1−φ)·d+θ=x″−d_(inf)are obtained.

It should be noted that |d_(inf)+d_(sup)|=2d.

The second set of pixels determined at step 120 is then for example:

-   -   the pixel at coordinates (E(x_(yn)),y_(n)),    -   the pixel at coordinates (E(x_(yn))+sign(d),y_(n)),    -   the pixel at coordinates (E(x_(yn+1)),y_(n)+1),    -   the pixel at coordinates (E(x_(yn+1))−sign(d),y_(n)+1),    -   where sgn(x) is the sign function of x

${{sgn}(x)} = \left\{ \begin{matrix}{- 1} & {{{if}\mspace{14mu} x} < 0} \\0 & {{{if}\mspace{14mu} x} = 0} \\1 & {{{if}\mspace{14mu} x} > 0}\end{matrix} \right.$

In the example of FIG. 3, the pixel at coordinates (E(x_(yn)),y_(n)) isthe pixel at coordinates (x_(m)+1,y_(n)), the pixel at coordinates(E(x_(yn))+sign(d),y_(n)) is the pixel at coordinates (x_(m)+2,y_(n)),the pixel at coordinates (E(x_(yn+1)),y_(n)+1) is the pixel atcoordinates (x_(m)−1,y_(n)+1) and the pixel at coordinates(E(x_(yn+1))−sign(d),y_(n)+1) is the pixel at coordinates(x_(m)−1,y_(n)+1).

With reference again to FIG. 1, the next step is step 130 that consistsin calculating the value of the output pixel P_(out) at coordinatesx_(out) and y_(out) in the output picture from the value of the pixelsof the set determined in step 120. For this calculation step, theparameters α and β are defined as follows:

α=|E(x _(yn))−x _(yn)|

β=|E(x _(yn+1))−x _(yn+1)|

The value of the pixel P at coordinates (x_(out), y_(out)) in the outputpicture is then calculated using the following formula that correspondsto a bilinear interpolation:

I _(out)(x _(out) ,y _(out))=(1−φ)·└(1−α)·I _(in)(E(x _(yn)),y_(n))+α·I_(in)(E(x _(yn))+sgn(d),y_(n))┘+φ[(1−β)·I _(in)(E(x_(yn+1)),y_(n)+1)+β·I _(in)(E(x _(yn+1))+sgn(d),y_(n)+1)]

-   -   where −I_(in)(x,y) designates the value of the pixel at        coordinates (x,y) in the input picture and I_(out)(x,y)        designates the value of the pixel at coordinates (x,y) in the        output picture.

The calculation is carried out on each video component of the pixel. Thedistance d calculated for the luminance video component is used for thecalculation of all the video components of the pixel from the outputpicture.

Other formulas for the calculation of the video components of pixels ofthe output picture can be considered. In the example above, theinterpolation is bilinear. The use of a linear interpolation or aninterpolation by selection of the closest neighbour or a combination oflinear or bilinear interpolations can be envisaged. These interpolationsare well known by those in the profession. Use of a greater number oflines or columns can also be envisaged. Finally changing theinterpolation formula according to the distance module d can be providedfor.

FIG. 4 illustrates a 400 device capable of implementing the method ofthe invention. Y_(in), Cb_(in) and Cr_(in) designate the videocomponents of the picture applied at the input of the device andY_(out), Cb_(out) and Cr_(out) designate the video components of theoutput picture of the device. These components are expressed in thecolour space [Y,Cb,Cr]. Naturally other colour spaces can be used suchas the colour spaces [Y,U,V] or [R,G,B]. For components expressed in thecolour space [R,G,B], a luminance component Y is recalculated todetermine the local gradients required for the distance calculation. The400 device comprises:

-   -   a calculation circuit 410 of distances representative of the        edge orientations at points of the input picture, this circuit        implements step 110 of the method of the invention, this circuit        delivers a distances chart, and    -   an interpolation circuit 430 implementing steps 120 and 130 of        the method of the invention from the input picture and the        distances chart delivered by the circuit 410, this filter        delivers the output picture.

FIG. 5 illustrates a second embodiment of the invention. This secondembodiment comprises the steps 501, 510 520 and 530. The steps 510 to530 are identical to steps 110 to 130 of the first embodimentillustrated by FIG. 1. This second embodiment comprises an additionalstep with the reference 501, of filtering of the input picture. Thepurpose of this filtering is to improve the calculation of the distancesof the edge orientations of the input picture. The filter used is basedon a Gaussian kernel. This filter is applied to the input picture signalluminance component Y. Hereafter, I_(jf) designates the component I_(j)filtered. The filtered component is then:

I_(f)(x, y) = G(x, y, σ) * I(x, y) = G(μ, v, σ) ⋅ I(x − μ, y − v) μ vwhere:${{G\left( {x,y,\sigma} \right)} = {\frac{1}{2\; {\pi\sigma}}{\exp \left( {- \frac{\left( {x^{2} + y^{2}} \right)}{2\; \sigma}} \right)}}},{\sigma^{2}\mspace{14mu} {is}\mspace{14mu} {the}\mspace{14mu} {variance}\mspace{14mu} {of}\mspace{14mu} {the}\mspace{14mu} {Gaussien}\mspace{14mu} {{kernel}.}}$

The filter used is for example the following 5×5 coefficients filter:

${\frac{1}{100} \times {\begin{bmatrix}1 & 2 & 4 & 2 & 1\end{bmatrix} \otimes \begin{bmatrix}1 \\2 \\4 \\2 \\1\end{bmatrix}}} = {\frac{1}{100} \times \begin{bmatrix}1 & 2 & 4 & 2 & 1 \\2 & 4 & 8 & 4 & 2 \\4 & 8 & 16 & 8 & 4 \\2 & 4 & 8 & 4 & 2 \\1 & 2 & 4 & 2 & 1\end{bmatrix}}$

FIG. 6 illustrates a 600 device capable of implementing this secondembodiment of the method of the invention. Y_(in), Cb_(in) and Cr_(in)designate the video components of the picture applied at the input ofthe device and Y_(out), Cb_(out) and Cr_(out) designate the videocomponents of the output picture of the device. Naturally other colourspaces can be used such as the colour spaces [Y,U,V] or [R,G,B]. Forcomponents expressed in the colour space [R,G,B], a luminance componentY is recalculated to determine the local gradients required for thedistance calculation. These components are expressed in the colour space[Y,Cb,Cr]. The 600 device comprises:

-   -   a first filter 601 to implement the first filtering step 501,        this filtering is applied to one or to all video components of        the signal of the input picture,    -   a calculation circuit 610 of distances representative of the        edge orientations of the input picture, this circuit implements        step 510 of the second embodiment of the method of the        invention, this circuit delivers a distances chart, and    -   an interpolation circuit 620 implementing steps 520 and 530 of        the method of the invention from the input picture and the        distances chart delivered by the filter 610, this circuit        delivers the output picture.

In the method of the invention previously defined, the distancecalculated at step 110 or 510 is used as is to determine the pixels ofthe second set. As a variant, if the distance calculated exceeds apredefined maximum value, this distance is capped at this maximum value.This maximum value is for example equal to 4 pixels for an SD (StandardDefinition) picture. In fact, beyond this maximum value, the value ofthe distance d cannot be low thus introducing errors into theinterpolation. According to another variant, if the distance calculatedexceeds this maximum value, the distance is adjusted to a null value.This second variant enables even further reduction of the risk of errorswhile being however less precise in the interpolation of some edges.

Naturally, the invention is not limited to the aforementionedembodiments. Any format conversion method or device in which distancesrepresentative of the edge orientations are calculated at points of theinput picture independently of the grid of the output picture, eachpoint being associated with a zone of the picture, and in which thevalues of pixels in a zone of the output picture are calculated from thedistance calculated for this zone and from the position of these outputpixels with respect to the point of the input picture falls into thefield of the present invention.

1. Method for converting the format of an input picture to an outputpicture, wherein it comprises the following steps: calculate, in atleast one point situated inside a zone delimited by a first set ofneighbouring pixels of said input picture, a distance representative ofan edge orientation at said point, determine, for at least one pixelsituated in a zone of the output picture corresponding to the zonedelimited by said first set of neighbouring pixels of said inputpicture, a second set of pixels in said input picture from thecalculated distance for said point in the zone and from the position ofsaid pixel of the output picture in said zone of the output picture,determine the value of said at least one pixel of the output picturefrom the values of the pixels of said second set of pixels.
 2. Methodaccording to claim 1, wherein said neighbouring pixels of said first setbelong to at least two consecutive lines or columns of pixels of theinput picture.
 3. Method according to claim 1, wherein the calculationof the distance representative of the edge orientation is carried out ata single point of said zone of the input picture.
 4. Method according toclaim 3, wherein said point is situated at the centre of said zone. 5.Method according to claim 2, wherein the distance representative of theedge orientation at a point of the input picture depends on a localgradient of the luminance video component of the input picture at saidpoint.
 6. Method according to claim 2, wherein, if said calculateddistance exceeds a predefined maximum value, said distance is capped atsaid maximum value.
 7. Method according to claim 2, wherein, if saidcalculated distance exceeds a predefined maximum value, said distance isadjusted to a null value.
 8. Method according to claim 2, wherein thevalue of said at least one pixel of the output picture is determined bybilinear interpolation of the pixels of said second set.
 9. Device forconverting the format of an input picture into an output picture, theinput picture and the output picture comprising pixels, wherein itcomprises: a calculation circuit to calculate, in at least one pointsituated inside a zone delimited by a first set of neighbouring pixelsof said input picture, a distance representative of an edge orientationat said point, and an interpolation circuit to determine, for at leastone pixel situated in a zone of the output picture corresponding to thezone delimited by said first set of neighbouring pixels of said inputpicture, a second set of pixels in said input picture from the distancecalculated for said at least one point of the zone and from the positionof said pixel of the output picture in said zone of the output picture,and to determine the value of said at least one pixel of said outputpicture from the value of the pixels of said second set of pixels.